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Abstract: Alberta Health Services (AHS) was awarded a grantfrom the Alberta Ministry 
of Human Services to promote applied mental health research within areas ofinterest to the 
Ministry. The grantfunded the Collaborative Research Grant Initiative: Mental Wellness 
in Seniors and Persons with Disabilities (CRGI), designed to collaboratively generate 
knowledge to improve the effectiveness of programs, services, and supports to the mental health 
needs of seniors and adults with disabilities. CRGI researchers received support via workshops 
that provided grant process information and basic research instruction, as well as an annual 
conference to share research results and network and connect with each other. We used social 
network analysis to assess whether CRGI objectives regarding collaboration were achieved 
and to provide examples ofpossible future strategic planning. A survey ofgrant recipients 
with questions addressing collaboration before and after CRGI events was also completed. 
Awareness among grant recipients increased drastically from before and afier the CRGI. 
Information workshops and the conference both provided successful support and increased 
awareness ofongoing work among grant recipients. 

Keywords: Social network analysis, collaboration support, knowledge exchange, research grant 
evaluation 

Introduction 

This project aimed to assess how a major collaborative research grant initiative affected interactions 
among grant recipients. The Collaborative Research Grant Initiative: Mental Wellness in Seniors 
and Persons ivith Disabilities (CRGI) was funded by a grant awarded by the Alberta Minister of 
Human Services to Alberta Health Services-Addiction & Mental Health. The CRGI had two 
main goals. First, it funded both academic and practitioner-driven research designed to assist 
individuals living with a mental illness and disabilities to maximize their independence in the 
community. In addition, CRGI was meant to increase awareness of Alberta-based research, and 
foster collaboration and knowledge exchange, among policy makers, researchers, and community 
agencies. Developing research collaborations across multiple organizations, disciplines, and 


The Journal of Research Administration, (46)1 




26 Barron, Scarlett-Furguson, Aspen 


locations is a complex challenge and requires significant support (Craven & Bland, 2006). 
Researchers have argued that we must demonstrate effective knowledge exchange practices 
in order to better understand what forms of support are effective under real world conditions 
(Norman & Huerta, 2006). 

The CRGI steering committee agreed that a first step toward enhancing collaboration within 
Alberta was to increase awareness among practitioners and researchers in the mental health field. 
Several information sessions and knowledge exchange events were held to support potential 
research grant applicants through the CRGI application process, to encourage collaboration, and 
to disseminate research results. Activities important to the Ministry included capacity building 
within community agencies funded by the Ministry; capacity building within the Ministry 
itself; and multi-sectoral collaboration. The CRGI steering committee also decided to evaluate 
the effectiveness of the workshops using social network analysis. This paper focuses on the social 
network analysis results, based on data collected from 26 CRGI participants who were surveyed. 
The project aimed to assess effects of the CRGI on collaboration between grant recipients and 
their knowledge of one another’s work. Therefore we also aimed to illustrate which components 
of the CRGI were affective in achieving the Ministry’s goals. The survey asked grant recipients 
for demographic information, through which CRGI activities they had met, and about their 
collaborative activities with one another before and after winning a CRGI grant and engaging 
in related events. We include information regarding support activities below and further details 
regarding the grants, projects awarded funding, and the activities that were undertaken to support 
their collaboration are available at the Alberta Addiction & Mental Health Research Partnership 
Program website (http://www.mentalhealthresearch.ca/KeyInitiatives/ResearchGrants/Pages/ 
default.aspx). 

Social Network Analysis 

Social network analysis (SNA) is the study of the structure of relations between actors, people, 
or organizations who have the capacity to take action (Wasserman & Faust, 1994; Scott, 2000). 
Key SNA principles include: actors are interdependent; resources such as information can be 
transferred between actors via the nature of their relationships; the form that relationships take 
between actors can limit or enable the capacity for individual action and; models of networks— 
their structure—are considered as regularly occurring patterns of relationships between actors 
within a network (Wasserman & Faust, 1994). In SNA, structure is often illustrated using 
sociograms or graphs where points (typically actors) are connected by lines (relations) (Scott, 
2000). In other words, a sociogram is a graph that represents the people within a social structure 
and their connections to their peers. These graphs can be effective in visualizing what a network 
looks like by allowing relations to become visually observable in situations where the observation 
is typically theoretical or ephemeral. 

Social network analysis is a useful means to examine interdisciplinary collaboration (Stokols 
et al„ 2003; Godley, Barron, & Sharma, 2011; Godley, Sharkey, & Weiss, 2013). For example, 
a comparison of individual attributes and network structure can provide valuable information 
regarding changes in interdisciplinary cooperation over time (Aboelela, Merrill, Carley, & Larson, 
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2007). Social network analysis can also be applied to facilitate collaborative innovation and 
knowledge exchange by elucidating structure and qualities of collaborator networks (Hargadon, 
20020), examining how particular characteristics of collaborations affect performance (Thomas- 
Hunt, Ogden, & Neale, 2003), or revealing forms of collaboration between actors most likely to 
be productive and maintained (Provan, Harvey, Guernsey de Zapien, 2005; Long, Cunningham, 
Wiley, Carswell, & Braithwaite, 2013). 

For example, Aboelela et al. (2007) used SNA to evaluate a Centre for Interdisciplinary Research 
on Antimicrobial Resistance (CIRAR). They designed questionnaires to ask participants about 
their knowledge of others involved in the collaboration, as well as the types of work they engaged 
in with each other. Surveys were administered at the beginning of the group’s formation and at 6 
and 12 months afterward. By examining changes in disciplinary network composition over time, 
Aboelela et al. (2007) were able to determine whether the goal of facilitating interdisciplinary 
research was accomplished. Results demonstrated that after one year of CIRAR’s existence, 
the network ceased to be dominated by participants from medicine and nursing and became 
composed of a variety of other disciplines. Aboelela et al. (2007) found that the most productive 
network members were highly connected to others and positioned between many others 
(bridges), making them potentially good facilitators for the transfer of information and building 
new connections. 

While there have been a number of studies identifying leadership in networks (Long, et al., 
2013), exchange of specific information (Wensing, Lieshout, Koetsenruiter, & Reeves, 2010), 
or implementation of specific evidence based practices (Palinkas, et al., 2011), this current 
project provides evidence of effective support for enhancing awareness as a first step in building 
connections between otherwise isolated potential collaborators. As such, this paper provides 
useful insights for major research grant evaluation, knowledge exchange, and collaboration 
building practices. We assess CRGI member connectedness and awareness before the CRGI, 
determine which of the CRGI activities were effective at enhancing awareness of research among 
CRGI research grant recipients (if any), and describe what changes in networks occurred in 
relation to the CRGI. The research questions we used to guide this process are: 1) How well 
integrated were CRGI network members prior to the CRGI-related activities?; 2) Did CRGI 
activities affect integration?; 3) What changes in relationships among network members occurred 
after CRGI related activities? 

Research Methods 

In order to examine the links and relationships between members of a bounded population it 
is important to survey as complete a network as possible (Heath, Fuller, & Johnston, 2009). 
Missing responses from central or influential members of a social network can result in a distorted 
representation (Quatman & Chelladurai, 2008). Hence, SNA research generally does not employ 
random sampling (Quatman and Chelladurai (p.348, 2008). Nevertheless, survey data are 
typically used for SNA and have been found to produce reliable results (Marsden, 1990). We 
used purposive sampling in our surveys to ensure that all grant recipients possessed the required 
characteristics for the purposes of our study (Cohen, Lawrence & Morrison, 2007). CRGI 
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research grant recipients were located throughout the province of Alberta and were accustomed 
to electronic communication with the grant administrators. Consequently, it was most practical 
to conduct an online survey. Survey data are typically used for SNA and have been found to 
produce reliable results (Marsden, 1990). 

Within the CRGI there were 37 research grant recipients. Three grant recipients were excluded 
from this study due to illness, maternity leave, or retirement. This is not a detrimental flaw, since 
our goal was to evaluate CRGI activities for all active grant recipients. Of those 34 individuals, 27 
research grant recipients provided consent to participate in the study. One person who consented 
subsequently did not respond, leaving us with 26 study participants and a response rate of 76.5%. 
Our response rate is considered acceptable for SNA and is high enough to limit effects of missing 
data (Wasserman & Faust, 1994). Moreover, we tracked which grant recipients participated in our 
study and to our knowledge no particularly influential or central research grant recipients did not 
participate. Half of the study participants were male and half were female. They were generally aged 
40 or older, with the majority being primarily involved in mental health as researchers, academics, 
or service providers. See Table 1 for the full demographic profile of participants. 

Network studies with a small sample 
size run the risk that study participants 
can be identified when attributes are 
included because readers can combine 
relationships illustrated in sociograms 
with attribute data and publicly available 
information (e.g., via websites). We 
mitigated this problem by including 
attribute data in the aggregate in our 
tables, and only illustrating gender and 
career stream within mental health 
related work. We did not identify 
the specific organization with which 
our participants are affiliated. Study 
participants were informed of the risks 
associated with participation in this study 
and ethics approval was obtained from 
the former Community Research and 
Ethics Board of Alberta (CREBA). 

Studyparticipants are located throughout 
the province of Alberta and are familiar 
with electronic communication with 
the grant administrators. Because of 
participants geographically distributed 
work it was practical to conduct an 


Table 1. CRGI grant recipients, descriptive statistics 


Characteristic 

Frequency (%) 

Gender 

Male 

13 (50%) 

Female 

13 (50%) 

Age 

29 years or younger 

0 (0%) 

30-39 years 

3 (12%) 

40-49 years 

6(14%) 

50-59 years 

10 (40%) 

60 years or younger 

6 (24%) 

Primarily involved in mental health work as 

Researcher 

14 (54%) 

Administrator 

1 (4%) 

Service Provider 

8 (31%) 

Policy/Decision Maker 

1 (3%) 

Others (Manager, Quality 
Practice Lead) 

2 (8%) 

Affiliation 

Community Agency 

7 (33%) 

Academic 

14 (67%) 
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online survey. Participants were asked to indicate their relationship with each of the other research 
grant recipients out of a possible number of 27 (see Appendix A). Four levels of relationship 
were examined, whether he/she had heard of, knew the work of, had worked with, or had met 
with others. Respondents who indicated that they had met or collaborated with another grant 
recipient were prompted to specify how and a list of options were provided. Options for how 
they had met included: CRGI information sessions and workshops or an annual knowledge 
exchange conference. Options for whether they had collaborated included: whether they co¬ 
authored a paper, co-presented at a conference, shared knowledge, or won a grant together. The 
survey was administered after the final CRGI event, but grant recipients were asked questions 
related to relationships with fellow CRGI researchers both before as well as after the CRGI events 
were completed. Because of the short time interval from the CRGI events to the questionnaire 
forgetting of “before” and “after” relationships and cross contamination was minimized. 
Marin (2004) found that network survey respondents are more likely to effectively remember 
connections with individuals when they have had more types of relationships, and when they 
have known one another for longer periods of time. Participants who knew one another from 
“before” the CRGI are likely to have had longer lasting relationships and be more memorable to 
one another. Whereas respondents who met “after”—through the CRGI—had their memories 
reinforced by significant events such as: attending the CRGI workshops, winning the grant 
competitions, and attending the annual conference with its various networking events. Moreover, 
because our survey was with a whole network each respondent did not have to generate names 
through a free recall task, but were given a list of all possible names for each network question. 
All of these characteristics of our study contribute to confidence that respondent memory is not a 
confounding factor in our study. Respondents were also asked to identify how they were primarily 
involved in mental health work (see Table 1 above for participant demographics). In order to 
track who had completed the survey, grant recipients were also asked to report their gender, age, 
and name (Appendix A). 

Once data were collected they were exported from the online survey software and imported 
into UCINET for analysis (Borgatti, Everett, & Freeman, 2002). UCINET is software that was 
developed specifically for the purpose of analyzing network data. We used UCINET to examine 
density, degree, and isolates, a form of descriptive analysis that does not typically integrate 
common statistical procedures such as tests of significance. However, we also used UCINET to 
compare the differences in density before and after involvement in CRGI activities and events. 
UCINET uses a bootstrapping procedure to take multiple samples (5,000) from the data and 
then calculate a classical t-test of significance. This procedure is useful for determining changes in 
density among the same set of actors for the same set of relations over time. UCINET is also fully 
integrated with NetDraw software for visualizing networks. This allowed us to visually analyze 
graphs for patterns and changes in networks and to use these visualizations to reflect on post¬ 
evaluation strategic action. We did not consider direction of ties in our analysis, as some other 
SNA studies have done, because our focus was on the overall increase in ties for those who took 
part in CRGI activities. 
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Results 

Our first research question asked how well integrated CRGI network members were prior to 
CRGI related activities. To answer this question we examined the degree of overall relations in 
the network before the CRGI by looking at degree, isolates, and density of the network. These 
measures are illustrated in Table 2 for the ties “have heard of”, “knew their work”, and “have met”, 
as well as a multiplex network of all of these ties combined. Of particular interest is the tie “knew 
their work”, illustrated in Figure 1, because it is the most meaningful connection for potential 
future collaboration. One may have heard of, or have met another individual, but still be ignorant 
of their research. In the absence of knowing another potential collaborator’s work, one would 
be unlikely to approach them to collaborate on a project. For “knew their work” the network 
density was 3.7%, indicating that only 3.7% of all potential ties in the network existed at baseline. 
Fourteen individuals knew nothing of other research grant recipients’ work prior to CRGI 
(53.8%). The highest degree for the “knew of” network was 6, showing that the most connected 
person only knew the work of 6 other individuals. 


Table2. Density, isolates, and degree for CRGI networks before CRGI events 



Density % 

Isolates 

Mean degree 

Sd 

Min degree 

Max degree 

Have heard of 

8.6 

4 

2.154 

2.178 

0 

10 

Knew their work 

3.7 

14 

0.923 

1.385 

0 

6 

Have met 

7.7 

8 

1.923 

2.074 

0 

8 

Multiplex before 

0.185 

3 

4.615 

3.431 

0 

26 





Figure 1. Baseline Sociogram for “knew their work” Network 
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Our second research question asked if CRGI activities affected integration. Again, we examined 
degree, isolates, and density for the CRGI event networks. Opportunities to meet at the CRGI 
were organized around sharing information regarding grant recipient research and other related 
work. In order to clarify which of the CRGI events had been particularly useful in introducing 
grant recipients we asked those who met through our events to identify the event where they had 
met one another. Results for the CRGI workshops and annual conference events are provided 
in Table 3 and illustrated in Figures 2 and Figure 3, respectively. While the density of the CRGI 
workshops network was relatively low (5.5%) we can say the event was successful in that there 
were only 12 isolates, a combination of grant recipients who already had met prior to the sessions 
(as per research question 1), or at our other event illustrated in Figure 3. Similarly, the annual 
conference event was successful in bringing people together as the density increased over that of 
the workshops (7.4%), there were fewer isolates (4), and there was an increase in maximum degree 
from before and after the workshops. 

Our third and final research question asked what changes in relationships among network 
members occurred after CRGI related activities. It was asked to help clarify the overall success 


Table 3. CRGI event networks measures 



Density % 

Isolates 

Mean degree 

Sd 

Min degree 

Max degree 

CRGI information 
workshops 

5.5 

12 

1.385 

2.114 

0 

8 

Found in translation 

4.7 

4 

1.846 

2.231 

0 

11 






Policy ^decision maker 
Community agency 

Other, quality lead policy/icsea 


8 


White 

Grey 



Diamond 


Figure 2. Sociogram for CRGI Workshop 
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Figure 3. Sociogram for CRGI Conference 


of CRGI efforts to introduce grant recipients to each other and to increase awareness of their 
work. Network measures for research question 3 are displayed in Table 4 and illustrate that there 
was a sharp increase in all network measures of inter-grant recipient connection and a decrease 
in the number of isolates. After the CRGI workshops and conference all grant recipients had 
at least heard of one other grant recipient, at least one individual had heard of nearly all of the 
grant recipients, and density increased to 22.8% (from 8.6%). For the “have heard of” network, 
22.8% of all possible ties had been made (not illustrated). For the “had met” network, one isolate 
remained, but 13.8% of all possible introductions had been made, an increase of 6.1% over the 
pre-CGRI situation (Figure 4). For the “know of” network one isolate remained, and 16.6% of 
all possible connections had been made, an increase in density of 12.9% over baseline (Figure 5). 

Finally, the multiplex network is composed of all possible ties and demonstrates that 47.4% of all 
possible ties had been made (an increase in density of 47.2%); at least one person had been tied to 
every other possible individual in some way (Figure 6). Examining Figure 6 and Figure 7 provides 
a comparison of all possible ties before and after the CRGI (respectively) and illustrates the 


Table 4. Post-CRGI event network measures 



Density % 

Isolates 

Mean degree 

Sd 

Min degree 

Max degree 

Have heard of 

16.6 

i 

4.154 

4.312 

0 

21 

Knew their work 

13.8 

i 

3.462 

3.066 

0 

14 

Have met 

22.8 

0 

5.692 

5.312 

1 

23 

Multiplex before 

0.474 

0 

11.846 

6.0363 

4 

25 
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significant increase in number of connections between baseline and conclusion. All differences 
in density from baseline were statistically significant (p < 0.05). Our results demonstrate that 
overall, the CRGI was successful in achieving its goals of increasing awareness of Alberta-based 
research among our grant recipients. 



Figure 4. Sociogram for “have met” After CRGI Events 
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Figure 7. Sociogram for All Relations Before CRGI Events 
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Discussion and Conclusions 

Opportunities for research grant recipients to meet face to face were organized around sharing 
information regarding grant recipient research and other work. Our survey results regarding 
which activities may have affected integration (Table 2 and Figures 2 & 3) demonstrate that the 
events were successful and the annual conference further enhanced the relationships between 
grant recipients overall. 

Research administrators interested in using information derived from network analysis will find 
it is most useful when combined with a strategic vision of organization goals. Beyond merely 
tracking progress toward those goals, it can also be used to identify strengths or, alternatively, 
areas in need of improvement in order to adapt activities toward achieving objectives (Aboelela, 
Merrill, Carley, & Larson, 2007). We provide an example of such network engineering by focusing 
on the objective of building more meaningful relationships between research grant recipients. 
Second, we use our network graphs to demonstrate how to identify key network members and 
how we might have used information about differences between types of relationships in order to 
achieve organization objectives. 

One of the primary goals for the CRGI was to increase meaningful relationships among mental 
health researchers in Alberta. While this study has demonstrated that this goal was achieved, 
it would be possible to use these analyses in order to further engage in strategic intervention to 
build even more meaningful relationships between its grant recipients over time. For example, we 
recommend finding individuals who are connected through “have met” or “heard of” and further 
connecting them through knowledge of one another’s work (Norman & Huerta, 2006). Another 
approach would be to determine individual relationships in order to pinpoint which people might 
play a key role in implementing research based practices, but this would not be possible in the 
absence of any knowledge regarding what research is available for translation (Palinkaset ah, 2011). 

Research has demonstrated that individuals with a high degree of centrality can be useful in 
taking on leadership positions (Long, et ah, 2013). Thus, well connected individuals can assist 
in building future collaborations and sharing important information. The “know of” sociogram 
(Figure 5) is of particular interest to our potential strategic planning for network engineering. In 
our “know of” network, the research grant recipient represented by the gray square in the center 
of the sociogram is a “star” in that the person knows the work of many other individuals, including 
service providers and researchers. Relative to having heard of, or met others, knowing others’ work 
is a particularly strategic relationship in that this knowledge is important for identifying common 
interests among our grant recipients. This research grant recipient could play a particularly 
important role identifying potential future collaborators and facilitating their work. Figure 5 
also contains other valuable information. This grant recipient’s occupation is as a service provider 
which in combination with her knowledge of research and connections to other researchers or 
service providers can allow her to play an important role in moving research to practice. Future 
work with the CRGI could also examine the networks for betweeness centrality, which provides 
a measure of the shortest path between actors and may indicate the most effective means of 
brokering new relationships. Godley, Sharkey, and Weiss (2013) used these network measures to 
identify leaders in networks of neuroscientists at a university brain research institute. A research 
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administrator could easily make use of such information to broker relationships between leaders 
and other network members, to disseminate information, or to achieve other goals. 

Our aims for this study were to use social network analysis to review the success of a collaborative 
research grant initiative in meeting its goals of increasing research awareness and connecting 
grant recipients from diverse disciplines and organizations across Alberta. We also provided 
some examples of potential strategies for future network building. Both the CRGI workshops 
and annual conference events helped facilitate Ministry goals. Research administrators will likely 
find that building support into research grants, such as workshops that provide an overview of 
available grants and the application process or conferences that incorporate networking activities, 
will likely enhance research awareness and collaboration in other collaborative grant initiatives. 

Some limitations of this study include that our survey was administered at one point in time, 
rather than at several time points. Having measures at more than one point in time may have 
allowed us to provide a more nuanced analysis and reduced the possibility of memory failure 
in our participants. Administering the survey at only one time point was due to our funding 
requirements. The survey had to be administered before the end of the grant term and two more 
networking events occurred after our survey. Our results likely under estimate the degree to which 
the CRGI increased awareness and collaboration among grant recipients. Moreover, we were 
unable to follow up and test future strategic planning for our research networks and we could 
not examine the specific types of collaborations that resulted from the CRGI. Additionally, we 
were only able to achieve a response rate of 76.5%. For network analysis, having full member 
participation is important for understanding network dynamics. Missing a key influential 
individual may also have implications for using SNA for strategic planning. However, in our study 
we did know all potential participants and to the best of our knowledge none who were particularly 
influential or central did not participate. Future research should examine long term outcomes 
of collaborative research grant initiatives. Also, whereas we examined a broad network building 
intervention—the annual conference event—future research might more closely examine specific 
elements of such an event to consider their individual utility. Despite these limitations, our results 
demonstrate that overall, the CRGI was successful in achieving its goals of increasing awareness 
of Alberta-based mental health and disability research among our grant recipients and bringing 
them together to foster future collaboration. In addition, our results demonstrate the value of 
social network analysis for evaluation of the effectiveness of CRGI type initiatives. 
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Appendix A - Network Survey 
MODIFIED ONLINE SURVEY 

Please indicate whether you have “heard ofT “know,” “met,” or “collaborated with” for each of the 
individuals listed for each scenario. 

Think about before you appliedfor a CRGI Research grant... 

1. Did not you know this person, skip to next person 

2. Had you heard of 

3. Knew their work 

4. Had met 

5. Had collaborated with 

a. Co-applied and won a research grant with 

b. Co-presented at a conference with 

c. Shared knowledge with 

d. wrote a journal article with 

e. Other_ 

Now think about today and afieryou received a CRGI Research grant... 

1. Do not know this person, skip to next person 

2. Have you heard of 

3. Know their work 

4. Have met 

5. If you’ve met, where? 

a. CRGI Information sessions/workshops 

b. Found in Translation 

c. Other_ 

6. Have you collaborated with them? 

a) Co-applied and won a research grant with 

b) Co-presented at a conference with 

c) Shared knowledge with 

d) wrote a journal article with 

e) Other_ 

— SBfflnternational 







